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'This  report  describes  the  results  of  the  program  of 
research  carried  cut  atj^^Hai^vardUnlversjlty  curing  tne  pericc 
Qptober  1,  1973  chroug*w\ucust  31,  1 975  uti 1 i zi ng  tne  LJP-1C 
and  various  resources  orh  the  ARPANET  during  that  perioc. 


The-vork  was  divided  i  nise  three  major  areas:  Automatic 
Programming  Research  and  Tools  for  Software  Production; 
Network  Research;  and.  Operating  System  Research. 

In  the  first  area,  our  major  results  have  been  related 
to  the  enhancement  of  the  ECL  Programming  System  and  the 
utilization  of  ECL  as  the  basis  for  tools  for  software 
production.  We  have  also  experimented  with  a  number  cf 
applications  of  cCL.  Finally,  a  number  of  efforts  have  been 
concerned  with  various  aspects  of  the  question  of  program 
optimization. 

-  During  the  period  considerable  use  has  been  made  of  the 
ARPANET  anc  a  major  programming  system  developed  (PhL  on  the 
POP- 11/45)  using  a  variety  of  tools  anc  resources  at  several 
si tes  on  the  net. 

Our  efforts  in  the  operating  systems  area  have  been 
rather  modest  with  work  concentratea  primarily  or,  cevisir.c 
system  architectures  for  higher  level  languages. 

Finally,  the  last  few  months  have  seen  the  beginning  cf 
design  work  on  various  cor.  ponents  of  a  Program  i  iani  pu  1  ati  cn 
System  —  a  system  devised  to  i ntegrate  the  various  tools 
for  program  development,  optimization,  and  mai ntenance. 
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SECTION  II 
INTRODUCTION 

This  report  describes  the  results  of  the  program  of 
research  carried  out  at  Harvarc  University  during  tne  period 
October  1,  1973  _  through  August  31,  .1975  utilizing  tne  j  DP-10 
and  various  resources  on  the  ARPANET  during  that  perioc. 

The  major  area  of  effort  has  been  that  of  developing 
tools  for  software  production  and  i nvesti gati ng  various 
aspects  of  automatic  programming,  particularly  those  having 
to  do  v/i  th  program  transf.crmati  on  and  optimization.  During 
the  period  the  ECL  system  has  been  considerably  enhanced  and 
a  number  of  new  tools  developed  based  on  ECL.  The  system 
has  also  been  used  for  production  problems  by  at  least  one 
government  agency. 

A  second  ar  a  of  Interest  has  been  in  the  use  of  the 
ARPANET  anc  various  resources  available  at  several  locations 
on  the  net.  Perhaps  the  major  accomplishment  here  was  in 
tying  together  the  resources  available  at  several  sites  to 
provide  the  tools  which  we  used  to  implement  a  major 
programming  system  —  a  full  PPL  system  for  the  PUP- 11/45 
computer. 

The  third  area,  that  cf  operating  systems  research,  has 
been  concerned  primarily  with  the  question  of  machine 
architectures  for  higher  level  languages. 

The  sections  following  are  devoted  to  the  various 
research  areas  in  which  ve  have  been  working  in  the  course 
of  the  contract  period.  In  each  we  have  attempted  to 
provide  a  brief  review  of  the  problems  wi  tii  wnicn  we  iiave 
been  concerned,  cite  the  accomplishments  during  the  contract 
period,  and  provide  reference  to  the  6i  b  1  i  o>'rapny  where  tne 
reports,  papers,  and  memoranda  which  contain  the  detailed 
results  are  listed.  .,e  v/ouJg  note  that  not  ail  cue  researc.i 
done  during  the  contract  period  has  been  written  up  at  this 
point  in  time;  it  is  our  hope  that  the  won.  wi 11  continue 
and  these  results  will  be  reported  in  the  coming  months. 
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SECTIOi:  III 

AUTOi  1ATIC  PRbGAA/IMING  AAL  TULLS  FOR  SOFTWARE  PRODUCTION 
1 .  Construe  t 1  on  of  Tools 
1.1  Development  of  the  ECL  System 
General  Improvements 

As  part  of  the  general  effort  to  improve  the  usability 
of  ECL  for  other  tasks  in  this  contract,  a  number  of  new 
features  were  aodea  during,  this  contract  perioc.  The  error 
reporting  and  trapping  facility  was  greatly  expanded.  The 
number  of  specific  errors  vas  increased  from  fifty  to  one 
hundred  and  twenty-four.  Error  messages  of  varying  degrees 
of  verbosity  keyed  to  each  specific  error  were  put  into  an 
error  message  file,  expanding  the  text  of  the  error  messages 
from  about  5000  characters  to  23,000  characters,  ano,  fer 
each  error,  appropriate  user  accessible  parameters  are 
provided.  Gy  revealing  the  precise  nature  of  error  ,  these 
new  diagnostics  nave  greatly  facilitated  the  rapid  debugging 
of  programs.  In  addition,  they  have  been  of  great  ne I p  to 
those  just  learning  the  ECL  language. 

Direct  access  I/O  routines  were  added  to  the  system 
allowing  octh  random  reading  and  writing  of  given  bytes  of  a 
file.  These  routines  were  of  great  utility  in  allowin'.  the 
error  message  file  to  be  indexed  ano  the  specific  message 
for  a  given  error  to  be  rapidly  accessec.  Also,  by 
remembering  the  byte  position  at  which  routines  are  defined 
in  files  to  be  compiled,  it  has  now  become  possible  to 
segment  the  compilation  of  large  packages  efficiently.  The 
direct  access  1/^  features  were  also  of  great  use  in  the 
implementation  or  paged  procedures. 

In  addition,  the  Ca.SE  statement,  more  powerful 
selection  semantics,  parallel  DECLs,  and  j  .»uC  mode 
validation  were  aooed,  ano  those  features  documented  in  the 
new  ECL  manual  (.manual  ].  The  ECL  system  assembly  language 
code  was  modularized,  and  monitor  dependent  code  i  solaced  in 
one  moaulc  in  order  to  make  the  process  of  producing  a  pure 
TE..EX  version  of  ECL  in  the  future  easier.  A  document 
describing  the  internal  representation  of  ECL  data  types  was 
produced  to  enable  better  mai ntenance,  anc  to  aid  in 
implementing  ECL  on  other  r. achines  [Conrad  Ij. 

(a)  The  ECL  Compiler 


Trie  ECL  compiler  has  been  tuned  and  i  mprovt'd 
during  tne  contract  period.  Its  new  features  are 
principally  documented  in  [manual]  anc  [upt-finuij,  so 
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they  will  only  be  sunr.arized  here. 

A  number  of  compiler  options  have  been  acced  to 
increase  civ  efficiency  of  compiled  coce  in  situations 
where  critical  simplifying  assumptions  about  the 
program  or  its  cata  are  not  otnerwi se  available.  For 
example,  expression  data  (list  structures  chat  c<-n  be 
both  manipulated  and  evaluated)  are  very  common  in  £CL; 
however,  they  must  normally  remain  uncompilea.  A  means 
has  now  been  provi aed  for  specifying  which  expression 
data  values  will  not  te  modified  and  may  therefore  be 
compiled.  As  another  example,  options  now  make 
possible  the  elimination  of  run-time  checks  curing 
access  of  data  components  and  pointer  referents,  so 
that  thoroughly  debugged  production  programs  are  not 
burdened  with  redundant  type  checking. 

A  variety  of  declarations  about  free  variables  may 
be  used,  causing  them  to  be  "frozen"  to  given  constant 
or  shared  values,  or  else  providing  information  about 
their  modes  and  dynamic  behavior.  Cither  declarations 
allow  the  programmer  to  control  the  accessibility  of 
local  and  global  names  in  his  production  package, 
creating  precisely  tne  environment  he  wants  its  user  to 
see. 


Two  methods  now  exist  by  which  procecure  calls  may 
be  eliminated  during  conpi lati on  for  efficiency.  The 
call  nay  be  replaced  by  on  expression  derived  from  the 
procecure  bocy  by  substituting  actual  express!  or. s  for 
formal  parameters.  Cr  a  user-defined  routine  t;u.y  be 
invoked  during  compilation  to  supply  a  replacement 
derived  from  the  given  actual  arguments.  These 
substitution  features  are  useful  for  efficient 
integration  of  user-cef  i  r.ed  mode  behavior  functions 
with  compiled  code. 

An  optional  optii  ization  phase  has  recently  peer. 
acGed  to  tne  compiler.  Tne  optimizer  is  designed  to 
remove  the  sorts  of  rcuuncancy  that  most  commonly  arise 
in  tCL  programs,  while  staying  within  the  strict 
resource  limits  tiiat  usually  govern  use  of  the 
compiler.  Although  LCL's  semantic  flexibility 
complicates  the  data  flow  analysis  problems  of 
reduncancy  removal,  the  optimizer  incorporates  a  simple 
data  invalidation  detection  scheme  that  enables 
significant  improvements  without  elaborate  analysis. 

The  user  interface  tc  the  compiler,  the  coi lection 
of  programs  that  help  the  prograi  ;mer  ceveiop  tne  <  icual 
declarations  tiiat  bine,  his  routines  into  a  prouuccicri 
pack  are,  h«.s  been  expanded  and  smocthec.  The  use  a 
cCL  s  program  edi  tor  in  tin's  package  Luildinc  process 
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has  permittee  many  features  that  i mprove  the  human 
engineering  of  the  interface  to  be  added  quickly  arc 
inexpensively.  Using  a  new  data  module  linking 
package,  facilities  have  been  added  that  per;.,i  t 
compiling  large  procrar;  in  limited  storage,  ei tner 
through  separately  corpilea  modules  1 i nkec  at  loading 
time  or  by  partitioning  the  program  eynani ca 1 1 y  curing 
compi 1  at i on. 

(b)  User-Defined  Data  Types 


The  notion  of  user-definec  data  types  was 
clarified  anc  strengthened  by  the  accition  cr  several 
features.  First,  user-defined  generation  functions  are 
now  used  consistently  throughout  the  system;  in  all 
instances  or  generation.  Second,  the  user  generation 
function  is  now  used  as  a  left  hano  conversion  function 
by  taking  conversion  to  be  a  case  of  generation  by 
example.  Tiii  rd,  through  the  accition  or  user-defined 
apparent  dope  vectors,  it  is  now  possible  to  model 
sequences  conveniently  even  thougn  the  actual 
implementation  is  done  with  objects  (such  as  pointers) 
which  have  no  dimensionality.  The  double  colon 
operator  vhich  allows  the  specification  of  extenaec 
modes  was  changed  to  take  keyuorcs  followed  by  an 
argument  to  allow  flexibility  in  cefining  the  behavicr 
of  these  modes. 

(c)  Free  Storage  management 


The  compact!  f  yi  r.g  garbage  collector  and  the 
"fasti  ode or  "frccuic  count"  storage  allocation  r.  ethcc 
made  possible  by  the  compact! fyi ng  garbage  collector 
became  an  i  rtcgrel  pare  of  the  ZCL  system  early  in  tic 
contract  period.  It  i  us  i  ecc.me  possible  to  ha;. cl  c  rue.; 
larger  problc-.  s  in  a  smaller  period  of  time  because  ti.e 
compact  i  fyi  n<  garbage  collector  enables  "r  ragi.cn  tat  ior.M 
of  storage  to  be  correctec  by  consolidating 
discontinuous  free  storage  regions  into  one  large 
contiguous  free  area,  because  o.  this  ability  to  unco 
the  damaging  effects  of  "fragmentation",  it  iu.s  been 
possible  tc  allocate  panes  of  core  storage  without 
doing,  a  garua^e  collection  every  time  Luc  storage  free 
list  becomes  empty.  7hi s  ability  to  allocate  steraje 
freely  without  dana<'i  r  r  side  effects  has  cut  typical 
garbage  col  lecticr  ti.e  fi'urcs  by  a  factor  cf  from 
five  to  ten  times  I  Cor. ran  2J. 

The  compact i fyi r  •  carbaoe  collector  has  also 
allowed  t < i e  acdibici  oi  cxp«neab  1  e  stacks  to  tm.e  LCL 
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system.  Cy  allowing  stack  overflows  to  be  trappec,  and 
the  stacks  to  be  lengthened  vi  thcut  losing  the  context 
of  the  program  being  run,  the  number  of  failures  of 
programs  to  run  because  of  stack  overflow  has  been 
greatly  reducec.  This  ability  of  programs  to  aajust 
their  stacks  as  they  run  is  particularly  important 
because  it  is  difficult  in  aevance  to  cetermine  the 
stack  requirements  of  a  particular  program  with  varying 
sets  of  data. 

A  CEXPP  (compilec  procedure)  paging  mechanism  was 
implemented  in  order  to  run  larger  programs  in  smaller 
core  sizes.  The  basic  method  usee  was  to  prepare  a 
small  "little  brother*-  for  each  CEXPrt  to  be  paged  which 
determines  whether  its  "big  brother"  is  alreacy  in  core 
or  whether  it  must  be  read  in  from  a  disk  file.  The 
"little  brother"  contains  the  file  name  ana  byte 
position  of  its  "bic  brother"  in  the  paging  file,  and 
Dy  use  of  the  recently  implemented  direct  access  1/u 
routines,  the  "big  brother"  may  be  rapidly  brought  into 
core.  Routines  to  flush  other  CEXPhs  not  currently  in 
use  and  to  add  the  freed  space  to  the  system's  free 
list  without  a  garbage  collection  were  also  written. 
This  paging  mechanism  has  been  successfully  used  to 
page  the  ECL  editor,  a  very  large  program,  which  only 
needs  small  subsets  of  its  many  routines  to  be  in  core 
at  any  given  time. 

Oespi te  initial  promise,  the  attempt  to  implement 
the  Bob row- he gb re i t  control  model  revealec  serious 
questions  relating  to  its  overall  effect  on  the 
efficiency  of  the  ECL  system.  Basically,  the  fact  that 
new  nomenclature  (named  variables)  may  be  introcuced  in 
any  ECL  statement  (as  contrasted  to  LISP)  points  to  the 
possiblity  of  very  high  overhead  if  the  Bcbrow-i.ecbrei  t 
control  model  is  directly  implemented.  The  matter  is 
treated  i  r.  more  cetai  1  in  the  reference, 
ob row-., enure)  t  Control  iiocel  iiecons i  cerec"  [Conrad 
3  j . 

(d)  SPECL  Compiler 


The  SPECL  (Syster s  Prograrnmi  r,g  in  ECL)  project  is 
interred  to  extend  the  use  of  ^CL  into  areas  normally 
reserved  for  so-eallec  "implementation  languages."  It 
is  a  uialcct  of  ECL  that  can  compiled  to  machine 
code  that  rune  without  the  support  of  ECL's  runtii.e 
facilities.  It  offers  tne  opportunity  to  "contract" 
ECL  fer  special  applications,  since  SPECL-procucec  cote 
can  me  auqnenteu  by  just  the  runtime  support  (such  as 
storare  management  or  I/u)  that  is  actually  neeciec. 
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SPECL  also  offers  access  to  the  implementation  cf 
operators  and  the  choice  of  underlying  data 
representat i ons  at  the  hardware  level.  For  example, 
suppose  a  programmer  wants  to  implement  doubly  linked 
lists  using  a  minimum  of  storage  for  the  links.  one 
trick  is  to  give  each  element  a  single  link  fielc 
containing  the  bitwise  exclusive-or  of  the  aocress  of 
its  predecessor  wi th  that  of  its  successor.  Given 
pointers  to  any  two  successive  elements  it  is  then  a 
simple  matter  to  move  forward  or  backward  along  the 
1  i  st. 


SPECL  is  ideal  for  such  an  application  since  it 
permi ts  the  user  to  manage  storage  as  he  chooses  and 
allows  him  to  give  machine  coae  aefi nit  ions  for  the 
necessary  pointer  operations.  Access  to  the  haraware 
level  is  isolated  in  code  generation  templates  called 
Compiler  Control  Expressions  (CCE).  CCE 's  tell  the 
compiler  how  to  impler.  ent  a  given  operator,  aeptnai  ng 
on  the  states  of  its  operands.  The  descriptions  cf 
operand  states  serve  as  goals  for  the  coce  generator  as 
well  as  enabling  ccce  selection.  The  CCE's  contain 
enough  information  abcut  register  use  and  side  effects 
that  optimization  of  expressions  containing 
user-spec i f i ec  operators  need  not  be  interrupted. 

The  coiiDiler  consists  of  three  phases.  The  first 
labels  the  prograr.  tree  with  temporary  storage 
requirements  (for  subtrees  free  of  common 
subexpressions  (C3E)}  in  a  manner  similar  to  the 
Sethi-U 1 1  man  method.  Properties  required  for  register 
allocation  and  assignment  are  attachec  to  the  trees 
principally  lists  of  variables  potentially  invalidated 
by  assi gnmerts,  variables  whose  addresses  may  be 
tracked  by  register  allocation  anc  a  variety  of  miner 
properties  necessary  for  code  generation  such  as  the 
number  of  locals  ceclarec  in  a  block.  An  initial 
pruning  cf  the  possible  ccce  generation  templates  takes 
p 1  ace  as  well. 

The  second  phase  operates  on  the  tagged  tree.  It 
reorders  computations  to  try  to  minimize  the  use  cf 
temporary  storage.  This  step  is  necessary  because  the 
Sethi-U 11  man  algorithm  coes  not  allow  for  CSE's.  The 
method  is  .touristic  anc.  incomplete  because  the 
minimization  problem  is  polynomical  complete. 

Register  allocation  and  assignment  are  also 
performed  curing  tut  second  phase.  Like  minimization, 
the  assignment  problei  is  polynomical  complete,  and  it 
will  be  handled  in ur i s t i ca 1 1 y. 
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The  third  phase  is  strai ght forward  coce 
generation. 

Starting  from  a  trial  ordering  of  the  expressions 
in  each  context  (strai ght- 1 i ne  program  section),  the 
temporary-mi ni mi zati  or  procedure  examines  the  variation 
in  storage  requirements  over  each  context.  The 
vicinities  of  peaks  in  the  requirements  are  scanned  for 
target  positions  that  meet  a  simple  numeric  criterion 
based  on  the  temporary  usage  and  result  size  of  the 
neighboring  computations.  Expressions  are  moved  to 
these  favorable  positions  in  order  to  reduce  overall 
storage  use.  Subtrees  iree  of  common  sub-expressi ons 
move  as  units,  subject  to  safety  contraints.  nest  cf 
the  time  used  by  this  process  is  actually  spent  in 
recalculating  the  temporary  requi rements  after  a  move. 

Register  allocation  begins  with  live-deac  analysis 
of  variables  and  cci.inon  sub-expressions  and  determines 
the  minimum  distance  to  next  use  for  these  items.  This 
information  suffices  for  optimal  register  allocation  in 
straight  line  code.  bhere  branches  are  involved, 
however,  a  simple  di stance- to-next-use  is  not 
designated.  Instead  the  allocator  must  compare 
distances  with  a  heuristic.  The  best  one  would  induce 
information  about  relative  probabilities  of  taking  one 
branch  over  another.  Lacking  such  information  the 
choice  may  be  made  on  a  simple  average.  once  the 
allocator  decides  which  portion  of  the  program  the 
assignment  process  must  determine  whi ch  register  for 
each  such  generation  of  a  variable  and  must  do  it 
consistently  at  branches  and  join  points  of  the 
program.  It  must  also  allow  for  occasional  specific 
register  assignments,  heuristics  are  neeced  to  reduce 
the  computation  from  a  "try  all  assignments"  approach. 
At  present,  we  have  only  simple  algorithms  for  this 
problem.  As  usual,  the  issue  is  the  trae'e-or/  between 
the  cost  cf  tiie  algoritnn  ano  the  improvements  in  the 
code  i t  makes. 

In  summary,  then,  SPECL  will  extend  to  the 
hardware  level  the  methodology  that  will  characterize 
code  improvement  at  higher  levels.  Users  will  be 
permitted  to  become  involved  in  the  optimization  of 
their  programs,  and  they  will  not  neec  to  forsake  gooc 
structure  to  achieve  highly  efficient  performance. 

(e)  ECL  Model 


The  original  thegbrei t  2j 
of  the  semantics  or  the  ELI 
giving  an  cLl-codeu  virsion  of 


one  upeaLee  definitions 
language  were  cerir.cd  by 
tiie  interpreter,  by  the 


fall  of  1  973,  major  revisions  in  the  ECL  system  had 
rendered  these  out  of  catt  as  well  as  incomplete.  In 
the  fall  of  1973,  work  was  started  to  cevelop  a  new 
node  1 . 


There  are  a  numptr  of  motivations  for  this  work. 
First,  as  was  the  case  in  the  two  theses  ci  ttn.  above, 
this  method  of  modelling  has  proved  to  be  valuable  botn 
in  development  and  dccumentati on.  Partial  rnoGels  have 
been  codec  on  several  occasions! 

1.  An  upcated  mocel  of  portions  of  the  interpreter 
was  prepared  for  instructional  purposes  curing 
fall,  1973. 

2.  The  revised  node  compiler  was  modelleo  [Conrad 

2]. 

3.  Modelling  was  usee  extensively  in  preparing  the 
revi sea  edition  of  the  ECL  programmer 's  manual 
[Manua  1  j . 

4.  The  parser  anc  parser  window  have  recently  been 
node  lied. 

In  the  first  two  instances,  the  main  purpose  was 
pedagogical.  In  the  seconc  instance,  hov-ever,  the 
ncdel  precede  the  implementation  of  the  revision  cf 
declaration  semantics.  Models  have  been  an  important 
guide  in  the  current  development  of  the  PDP-11/45 
i npl enentat i on  of  ECL. 

In  spring,  1574  v.e  started  a  long-term  effort  to 
develop  a  nccel  for  tie  entire  ECL  system;  not  just  for 
the  interpreter.  The  major  goal  of  the  model  is  to 
serve  as  a  guide  for  implementations  of  ECL  on  other 
hare. ware  systems.  Accor*  ingly,  much  of  the  cesi<n 
effort  in  tne  rote  1  i.as  ioen  tne  attempt  to  i.olate  as 
much  as  possible  tne  j achi ne-depenaent  aspects  or  the 
system  (as  ..as  Leen  corn  in  tire  maciri  ne  language  coce 
for  the  PCP-IC  implementation  of  tiie  sytem)  vniie,  at 
the  same  time,  preserving  maximum  reacability  cf  the 
code.  A  major  revision  of  the  implementation  strategy 
for  trie  model  tool,  place  i  n  spring,  1575  to  this  end. 

(f)  Eci tor 


The  ECL  list  structure  eci tor  (ELI  7)  is  an  in-core 
editor  whic.i  operates  on  tne  internal  representation  cf 
urcompiied  routines  arc  ur.evaJuotcd  data  (that  is,  upon 
forms)  ratiier  tear,  or.  character  strings,  as  co 
conventional  editors. 
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We  have  extended  the  editor  to  include  a  file 
editing  facility  (FIXIT).  FIXIT  allows  a  package 
consisting  of  one  or  rore  source  files  to  be  cefinec. 
It  loads  the  package  by  parsing  each  file  to  construct 
a  single  block  containing  all  of  the  commands  in  each 
source  file  in  the  package.  At  the  sane  tine,  it 
constructs  properties  for  each  cefinition  (that  is, 
assignment  command)  in  the  file  which  relate  the 
identifier  anc  form  which  is  bound  to  i t  at  top  level 
to  the  corresponding  source  file.  The  source  files  are 
similarly  related  to  the  package  so  that  several 
packages  may  be  in  core  at  the  same  time.  This  process 
is  equivalent  to  loaojng,  except  that  the  properties 
allow  the  file  to  be  reconstructec  for  output. 

A  routine,  FIX,  may  Le  used  at  any  time  curing 
debugging  t  apply  EDIT  to  any  routine  loaded  or  to  the 
form  representing  any  file  loaded  by  FIXIT.  In  the 
latter  case,  upon  exit  from  the  eci tor,  FIX  rebincs  the 
definitions  in  the  file  In  case  new  or  revised 
definitions  have  been  edited  into  the  file  form. 
Finally,  the  routine  ILiiiPE,  applieG  to  a  package  name, 
will  write  out  new  versions  of  any  files  in  the  package 
which  have  been  eci tec  or  any  of  whose  definitions  have 
been  edi ted. 

1.2  Prograrranle  Theorem  Prover 

A  number  of  issues  relevant  to  ceve loping  a 
programmable  theorem  prover  were  investigated,  including 
techniques  for  specifying  the  behavioral  properties  of  sore 
data  structure  in  an  implementation  i  ncepencer.t  manner  which 
allows  verification  cf  the  correctness  of  concrete 
implementations  cf  the  cata  structure.  These  results  are 
described  in  [Spi tzen  0  WegbreitJ. 

1.3  Xeasuremcnt  Package 


A  numuer  of  acditional  mechanisms  for  measuring  arc 
probing  ECL  programs  have  been  completed  during  the  contract 
period.  A  procedure  which  takes  a  specification  of  a  class 
of  data  objects,  that  is,  a  mode  specification,  anc  returns 
the  amount  of  space  on  a  PCP-10  an  object  from  that  class 
will  require,  according  to  the  ECL  data  packing  conventions, 
is  one  such  mechanism.  We  have  also  extended  the  basic 
metering  facility  to  permit  a  ( non-recundant )  measure  to  be 
mace  of  an  arbi trary  collection  of  statements  within  a 
procedure.  As  part  of  this  extension  we  can  now  protucc  a 
profile  di splay  cf  the  metering  results  for  greater 
readabi 1 i ty. 
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1.4  Closure  .".ochar.ism 


Procedure  c  1  osurc  [h'eebrei  t  1  ]  is  a  program 
transf crr.iati  on  that  replaces  a  procedure  ay  an  equivalent 
one  in  which  the  values  of  some  formal  parameters  an*-  free 
variables  have  eecn  fixed  and  corresponding  si  rr.pl  i  f  i  cat  i  or.s 
have  been  naae. 

It  is  a  kind  cf  optimization  that  becomes  increasingly 
relevant  as  we  learn  mere  about  “pref abr i catee  construction" 
techniques  for  building  large  systems  from  standard 
components  by  tailoring  them  to  special  needs.  It  is 
well -suited  to  program  development  by  stepwise  refinement, 
an  implementation  discipline  that  appears  to  enhance 
reliability  and  maintainability  of  programs. 

Just  before  and  during  the  early  part  of  the  contract 
period,  we  constructed  a  preliminary  procedure  closure 
mechanism.  The  intent  was  two-fold.  It  was  to  provide 
immediate  benefit  to  ECL  users  as  a  practical 
code- i mprovement  tool,  ana  it  was  to  give  us  insig..t  into 
the  nature  of  incremental  optimization  techniques.  As  a 
practical  tool,  that  preliminary  closure  program  was  not  a 
success.  It  provec  awkward  anc  very  inefficient  to  use.  It 
was  not  released  to  the  general  ECL  user  community,  anc  in 
its  steac  several  features  were  included  in  tne  compiler 
( the  ability  to  “freeze"  few  variables  to  constant  or  snarcc 
values  or  to  fix  their  mooes;  facilities  for  in- line 
procedure  expansion  anc  invocation;  redundant  expression 
optimization,  and  so  on). 

un  the  ether  hand,  our  experience  with  closure 
profoundly  shaped  our  views  cf  how  tne  programming  process 
can  be  automated.  It  has  lec  us,  in  fact,  to  try  to  solve 
the  problem  by  generalizing  it.  Some  of  the  key 
difficulties  were! 

(1)  heec  for  precise  user  guidance.  ms  a 
semi-auto;,  ati  c  tool,  closure  requires  teat  simplifying 
assumptions  ue  supplied  from  which  to  cerive 
specializations.  The  restriction  to  assertions  about 
the  free  variables  anc  formal  parameters  cf  an  entire 
procedure  was  too  severe.  in  practice,  one  needs  to 
guiac  the  simplifier  tc  particular  program  regions 
selected  by  pattern,  ay  identifier  scope,  by  the  data 
types  of  constructs  tc  be  affected,  an*,  the  like. 

he  nave  Degun  to  satisfy  this  need  for  precise 
guidance  in  program  transformation  uy  the  development 
of  a  i.turi  tc  Aecnar  i  s; ;  [Jam.esJ  anc  oy  recent 
improvements  in  tne  ECL  program  editor  tp.anuaij.  The 
Rewrite  mechanism  is  a  pattern-di rected  transformation 
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facility  chat  pcrr  i  ts  replacement  rule-:  to  be 
associated  v/i  th  the  lexical  scopes  icr  vuriaples. 
Rewriting  rules  relax  the  restrictions  mat  grocecure 
closure  imposed  by  genera  1  i  z<  re'  t<  e  net  i  or,  of  procedure 
definition.  They  can  enDccy  sinpli/yirg  as  motions 
and  they  will  (in  a  future  system)  tner. selves  uc  the 
object  of  simplifications.  Tne  i_CL  Loiter  is  also 
pattern-directed,  and  it  is  now  fully  progranmaL Je,  so 
that  user-defined  transf ormati on  routines  c.  n  ue 
applied  to  pattern-selected  program  regions. 

(2)  Meed  for  program  investigation  ai cs.  Even  though  a 
program  improvement  tool  nay  require  user  help  to 
achieve  significant  results,  it  shoulc  not  be  assur.ee 
that  the  user  is  intimately  familiar  with  the  program 
being  processed.  In  tne  important  case  that  a  library 
routine  is  being  tailored  to  special  requi  rer.  ents, 
quite  thc>  opposite  nay  me  true.  Matner,  facilities 
that  help  the  user  oecorne  familiar  with  a  program 
package  shoulc  be  provi  oct,  sc  ttiot  ne  can  oe  sure  his 
simplifying  assumptions  about  its  oenavior  are  va 1 i c. 

As  a  step  toward  such  a  facility,  we  have 
implemented  a  program  that  studies  free  variable 
reference  patterns  in  an  ECL  package  Luenscnj.  Tie 
information  is  displayed  in  a  skeletal  rencering  cf  the 
program  that  also  cxnibits  its  recursive  calling 
structure. 

(3)  heed  for  a  data  base  supporting  stepwise 
development.  Large-scale  software  seems  to  oc  most 
reliable  anc  least  expensive  when  it  is  prccucec  in 
careful  increments,  eacn  of  which  specializes  so/ e 
aspect  of  an  abstract  algorithm  into  more  concrete 
terms.  optimization  should  oe  involvec  at  eacn  step, 
or  vital  information  ray  !e  lost  in  trie  clutter  ci  lew 
level  detail.  if  tie  progre;  must  uc  reanalyze*  eacn 
tii.  e  an  optimizer  is  invoked,  ti.o  cost  is  lii.ciy  to  lo 
extremely  ai  gh  ana  cisciplinec  i ;  pi  er.cr.tat  i  or,  n«_y 
disintegrate.  Mather,  tae  optimizer  snculu  be  cl  1  e  to 
make  incremental  improvements  without  foil  reanalysis 
by  maintaining  a  data  base  of  information  about  the 
program  acqui red  through  user  assertion,  measurement , 
and  analysis.  Since  implementation  often  preacts  uy 
trial  and  error,  tne  coca  case  should  incluue  a  ni story 
of  the  dove i opment  anc  the  user  shoulc  oe  able  to  unco 
decisions,  revise  them,  anc  then  repeat  unafrecteo 
portions  of  the  i np 1 et on tat i on. 

(4)  j.ced  for  a  cor  non  store  of  <;t  neral-purpo:  e 
optimization  techniques.  <nc  closure  experiment  \  as  an 
attempt  to  nch?  ovo  a  iii  i  ted  class,  of  cone  i  mprovenent  s 
with  1  ini  tec  resource;.  it  oei.oris  t  m  to  tn<-  oiiticulty 
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of  trying  to  factor  global  program  analysis  into  a  set 
of  independent  tools  wi thout  gross  duplication  cf 
effort.  Instead,  autcnatic  programming  tools  should  be 
integrated  around  a  single,  general-purpose  ser antic 
analyzer,  so  that  all  can  share  a  common 
representation,  a  common  data  base,  and  a  store  of 
analytical  and  manipulative  expertise. 

To  answer  these  neecs,  a  new  program  manipulation 
system  is  uncer  developtent.  Its  basis  will  be  a  full 
Symbolic  Evaluator  for  ECL  programs.  This  analyzer  will 
maintain  a  Program  Qata  base  to  support  i ncremental 
optmization  and  to  preserve  the  refinement  history  of  the 
user's  program  as  its  implementation  progresses. 

These  facilities  will  not  be  cheap.  Deep  analysis  cf 
practical  programs  is  costly  to  obtain  anc  to  maintain. 
They  can  be  viewed,  however,  as  an  investment  whose 
dividends  will  be  the  relative  simplicity  of  ceding 
special-purpose  tools,  like  closure,  better  coce  generators 
for  ECL  and  SPE CL,  and  a  data  representati on  optimizer,  to 
the  kernel  system. 

2.  Add  1 ?  cat i ons 

2.1  Parallel  Processing 

The  SVhVcA  system  has  been  developed  during  the 
contract  period.  The  .SYbVEA  system  permits  a  hi gn  level 
spec! f i cat  ion  of  problems  in  concurrent  control  in  the  SAL 
specification  language,  which  has  been  designee  and 
i mpl enenteo.  The  SYhVEh  synthesizer  then  generates  ECL 
runnable  code  which  enables  concurrent  processes  to 
communicate  with  each  other  in  the  manner  specif ieo.  A 
technique  has  now  been  developed  whereby  the  correctness  cf 
the  speci f icat ion  can  be  verified. 

Currently  cot.e  generated  to  effect  the  synchre  ni  zat  i  on 
is  written  Tor  use  wi th  the  cCL  control  extension  facility 
(Prenner's  Cl),  ano  the  adaptation  for  a  cooe  generator  for 
rloare's  monitors  (CACA,  uct.  1974)  is  written  out  not 
implemented.  In  addition  c  cooe  generator  for  Uijkstra's  P 
and  V  semaphore  operations  is  being  developed.  Initial 
results  for  this  code  generator  are  very  encouraging;  in 
many  cases  generated  code  is  equivalent  to  that  wiiicn  would 
be  hand-wri tten. 

The  results  of  this  work  are  described  in  [Griffiths 

2  j  . 


2.2  and  2.3  Data  1'ype  Optimization 
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The  extension  to  ECL  to  accommodate  sets,  tuples, 
stacks,  and  queues  has  Leen  done.  The  extension  induces 
new  notational  facilities  (in  the  case  of  sets)  for  tore 
natural  specification  of  operations,  he  have  usee  the  tote 
behavior  facility  to  specify  the  inter.cee  behavior  of 
objects  from  t:ie  classes  of  data  objects.  In  the  case  cf 
sets  we  have  i  mp  1  ementec  several  ei  fferent  uncerlying 
machine  representations  for  objects.  In  each  case  we  have 
defined  the  necessary  primitive  operations.  he  have  not  yet 
solved  the  problem,  however,  of  (mechanically)  cnoosing 
efficient  representati ons  for  objects,  taking  into 
consideration  the  access  patterns  and  other  primitive 
operations.  (e.g.  insertions,  deletions)  on  tne  objects. 
This  is  a  more  difficult  problem  than  anticipated  though  it 
still  promises  high  pay-off. 


2.4  Compound  Objects  and  Operations 


We  have  developed  the  foundations  for  explorin'.  the 
customizing  of  general-purpose  algorithms  for  the  conain  of 
matrices.  We  have  implemented  a  package  of  basic 
general-purpose  operations  and  defined  a  collection  cf 
representations  for  matrices,  with  particular  concern  for 
sparse  matrices.  The  package  will  use  information  about 
algebraic  properties  (in  particular  algebraic  identities)  of 
he  matrices  and  do  son e  optimization  of  the  operations 
taking  into  account  these  properties.  It  will  also  cefine 
for  the  user  so.  e  non-Lasic  operations,  like  transitive 
closure,  permitted  by  the  properties.  The  package  includes 
a  number  of  notational  extensions  to  facilitate  the  user's 
writing  of  iterations  (e.g.  il[I,*j  to  specify  the  selection 
of  the  Ith  row)  and  partitions  of  a  matrix.  Several 
uncerlying  representations  h..ve  ween  implemented  fer  sparse 
matrices  and  wo  are  just  new  at  the  stage  of  consi coring  the 
interaction  among  representations  and  opt i mi zat i ons  over 
them. 

2.5  Resource  Management 


A  linkage  1 
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External  references  are  defined  to  the  compiler  by  a 
compiler  criver  list,  external  pairs,  which  contains  the 
icentifiers  and  notes  of  external  routines  anc/or  cata. 
Following  conpi  lati  on,  the  linkage  cunper  traces  all 
compiled  routines  in  orter  to  locate  ail  external  references 
in  the  compiled  code.  Tie  results  of  compi 1 ati on  are  then 
dumped,  together  with  tables  vnich  allow  the  linkage  loacc-r 
to  link  separate  1 y-compi  1 ec  mocules  to  each  other  ana,  wnere 
required,  to  routines  and  cata  which  are  alreacy  lcacec. 


The  link  dumper  may,  in  fact,  be  used  to  separate  a 
mass  of  loaded  routines  ana  data  into  separate  files, 
together  with  information  required  for  linkage  loading, 
independent  of  use  of  the  compiler.  Due  to  the  manner  in 
which  routines  anc  data  are  representec  in  core,  such 
separation  can  ot  effected  for  any  data  objects  which  are 
bound  to  icentifiers  at  top  level  or  on  the  name  stack. 
Thus,  for  instance,  if  a  cata  taole  consisted  of,  among 
other  things,  a  sequence  cf  routines,  different  routines 
might  be  cum.pcd  into  ci  fferent  files  with  the  resulting 
files  being  reloadable.  A  forthcoming  version  of  the 
package  will  extend  this  behavior  to  arbitrary  data  objects 
which  are  rot  necessarily  related  to  a  compilation. 


3.  Re 1  a tec  Research  Ef f or ts 
3.1  Property  Extraction 

Property  extraction  is  a  generic  name  for  a  set  cf 
techniques  for  efficiently  extracting  from  some  set  cf 
programs  a  collection  of  properties  which  enable  program 
transformation  ana  optimization.  During  the  contract  peried 
a  prototype  system  to  explore  the  technique  was  ceveloped 
anc  reported  in  [ocher  1 i sj.  i.ork  is  continuing. 


-A  .  .  >  A 
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SEC  TIER  IV 

DATA  STQA/GE  ARD  TAARSFER 

4.1  Use  of  the  DATACbiTPUTEh 


Experience  with  the  use  of  the  DFTP  (Dataconputer  File 
Transfer  Program)  developed  by  CCA  has  shown  that  a  more 
automatic  methoc  of  file  archiving  and  retrieval,  vhi le 
feasible,  is  probably  not  worth  the  development  effort 
reouirec.  he  have  also  investigated  the  feasibility  cf 
referencing  files  stored,  on  the  Dataconputer  from  ECL 
programs,  'while  this  facility  is  easily  i  rr.pl  ementab  1  e  in 
ECL,  given  recent  improvements  in  the  input-output  anc 
error-hand  1 i ng  facilities,  the  result  would  be  to  overburden 
the  currently  1  ini  tec  nur.ber  of  network  ports  on  CCA-TEi.ER 
at  very  little  gain  to  our  own  operation.  Therefore,  r,o 
actual  development  work  has  been  undertaken  in  this  area. 

4.2  Datacomputer  and  Datalanguage  Development 


Except  for  several  consultations  with  CCA,  no  work  has 
been  done  in  this  area,  blur  current  attitude  vi  th  regard  to 
conversion  of  data  stored  on  the  Dataconputer  is  that,  in 
the  absence  of  economic  justificatcn  in  the  case  cf  ary 
particular  application,  the  actual  computations  recuirea 
should  take  place  at  other  sites.  Such  conversions  may  Le 
done  with  significantly  less  overhead  by  a  facility  which 
knows  both  the  source  and  target  data  types  than  is  possible 
at  a  centralized  facility  which  must  deal  with  a  far  larger 
set  of  data  types  and  formats. 

4.3  Graphics  Studies 


Tiie  develop;,,  or,  t  of  a  sopni  sti  catec  grapi.ics  system  was 
unoertaken.  The  objectives  of  tin's  effort  were  to 
experiment  with  t,,e  viaoility  of  rrapnics  processors  with 
graphics  instructions  sets  capable  of  directly  interpreting 
a  high  level  data  structure  to  produce  graphical  output. 
This  approach  is  to  be  contrasted  with  the  classical 
real-time  compilation  f  a  data  structure  into  some 
low-level  object  format  suitable  for  execution  by  a  graphics 
control  1  or. 

A  system  th.3t  vou  1  c  support  research  in 
mi  cro-progran.ven  -rapines  instruction  sets  was  designed  arc 
partially  implemented.  It  consi stou  of  a  graphics  processor 
( i  u.;- 1  1 /4u }  linkec  to  a  iocai  processor  (PDF- 1  1/1  d  )  anc  a 
mb i'-10  iiost  machine.  the  f,)P-ll/4C  was  extensively  modified 
to  oe  a  eyr:a;.  i c.i t  1  y  .  i  cr< -pre  granmab  1  e  graphics  processor. 
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».e  addec:  a  wri teazle  control  store,  hardware  ini  cro-prograrn 
development  aids,  anc  (partially  implemented)  regi  sters 
accessable  at  t  e  micro- program  level  which  crive  display 
hardware.  A  special  interface  allows  the  two  PJP-lls  to 
share  the  same  aw.ress  space.  A  high-speed  parallel  channel 
was  built  to  permit  fast  access  to  the  PDP-1G. 

A  large  amount  of  software  development  stands  In  the 
way  of  the  relatively  snail  amourt  of  hardware  work 
necessary  to  complete  the  system.  The  investigation  was 
conducted  by  a  one-half  tine  research  assistant  until  his 
time  was  preempted  by  his  thesis  research.  The  project  is 
currently  inactive,  penai no  the  availability  of  students 
with  appropriate  interests  and  background. 
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SECTILli  V 

SYSTEM  ARCHITECTURE  FUR  HIGH-LEVEL  LANGUAGE 

The  problem  ot  I  nip  1  er.  ent  i  ng  complex  computet  i  oris  as 
embodied  by,  say,  high-level  languages  has  customarily  been 
attacked  in  relation  to  classical  machine  archi tectures. 
These  archi tectures  have  been  ana  persist  in  being  rather 
arbitrary  in  that  they  dc  not,  in  any  formalized  way, 
reflect  the  nature  of  the  computations  that  are  anticipatec. 

The  problem  of  constructing  machine  architectures  that 
are  appropriate  to  specific  computational  problems,  (in 
particular  ELI  programs).  Is  currently  being  i nvesti catec. 
The  objective  of  this  effort  is  to  aevelop  formal  methods  by 
which  one  may  take  a  representation  of  a  desired  computation 
and  transform  it  into  an  implementation  that  capitalizes  on 
the  specific  nature  of  the  computation. 

The  approacii  being  pursued  represents  the  computation, 
(e.g.  the  ELI  interpreter),  within  a  formalism  that 
introduces  no  overhead  ana  exhibits  all  potential 
concurrency.  This  representation,  resembling  an  infinite 
Petri  net,  is  then  subjected  to  a  series  of  transformations 
that  reduce  i t  to  a  finite  form,  and  in  particular,  a  form 
with  properties  raking  it  readily  i mp) ementab 1 e  by  some  set 
of  hareware  primitives.  Each  transf ormat i on  is  chosen  by 
the  system  designer  and  has  a  quantitative  effect  cn 
performance  that  may  be  readily  perceived. 

This  research  is  aimec  at  developing  a  system  that 
derives  increased  representational  anc  execution  efficiency 
from  the  fact  that  ELI  is  the  lowest  level  of  machine 
language  and  determining  the  facilities  and  character! sti cs 
such  a  system  should  possess  as  a  whole. 
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